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PREFACE 


This  paper  was  prepared  for  the  Deputy  Under  Secretary  of  Defense  for 
Environmental  Security  under  a  task  entitled  “Assessment  of  Traditional  and  Emerging 
Approaches  to  the  Detection  and  Identification  of  Surface  and  Buried  Unexploded 
Ordnance.”  This  paper  fulfills  subtask  h  of  that  task. 

We  would  like  to  thank  Tom  Altshuler  and  David  Sparrow  whose  comments  and 
suggestions  greatly  improved  the  quality  of  this  paper. 
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EXECUTIVE  SUMMARY 


A.  THE  REMOTE  MINEFIELD  DETECTION  SYSTEM 

Millions  of  acres  of  government  land  are  contaminated  with  unexploded  ordnance 
(UXO),  a  result  of  years  of  testing  and  training  in  the  armed  forces.  As  part  of  the  effort 
to  prepare  some  of  this  land  for  use  other  than  as  test  ranges,  programs  are  underway  to 
develop  methods  to  safely  and  reliably  detect  UXO  so  that  the  contaminated  sites  may  be 
cleaned  before  realignment. 

This  report  describes  the  REmote  Minefield  Detection  System  (REMIDS) 
developed  by  the  U.S.  Army  Engineer  Waterways  Experiment  Station  to  detect  surface 
UXO  and  reviews  the  performance  of  REMIDS  in  tests  at  the  Yuma  Proving  Ground, 
Arizona,  and  at  Fort  Rucker,  Alabama. 

The  principle  behind  REMIDS  is  to  enhance  the  discrimination  of  surface  UXO 
by  relying  on  multiple  signatures:  surface  UXO  may  exhibit  a  unique  combination  of 
reflectance,  polarization,  temperature,  and  footprint  (shape)  compared  to  natural  objects 
in  the  UXO’s  surroundings.  Discrimination  based  on  four  signatures  is  in  principle 
greater  than  that  based  on  fewer  signatures. 

The  REMIDS  hardware  consists  of  an  airborne  line  scanner  with  sensors  that 
measure  the  reflectance,  polarization,  and  thermal  response  in  710  round  “spots,”  each  of 
which  subtends  1.25  mrad.  (The  reflectance  and  polarization  sensors  are  active  and  utilize 
a  Nd:YAG  laser;  the  thermal  response  sensor  is  passive.)  The  710  spots  partially  overlap 
and  are  arranged  in  a  line  such  that  the  total  field  of  view  of  one  scan  line  is  1.25  mrad  by 
40  deg.  The  scan  rate  of  the  device  is  350  lines  per  second.  Thus,  when  the  REMIDS  is 
flown  in  a  helicopter  at  an  altitude  of  130  ft  and  a  speed  of  32  knots,  each  of  the  three 
sensors  will  digitize  its  own  map  of  the  ground  level  with  a  pixel  size  of  1.9  in.  x  1.9  in. 

Analysis  of  the  REMIDS  digitized  data  is  performed  in  three  stages.  In  the  first 
stage,  a  computer  classifies  each  pixel  as  from  a  UXO  candidate  or  from  background.  (At 
some  sites,  the  spectral  return  of  the  background  is  sufficiently  distinct  from  all  UXO  that 
this  stage  is  the  only  step  required  to  achieve  satisfactory  performance.)  In  the  second 
stage,  a  requirement  on  the  minimum  size  of  objects  (i.e.,  on  the  minimum  number  of 


contiguous  pixels)  is  imposed.  In  the  last  stage,  an  operator  views  all  potential  candidates 
and  determines  if  they  are  UXO  or  background.  The  operator  may  also  classify  the  type 
ofUXO. 

B.  ADVANTAGES  AND  LIMITATIONS 

The  REMIDS  technology  has  both  advantages  and  limitations  over  other  systems 
in  locating  UXO.  As  an  airborne  system  based  on  current  technology,  it  promises  several 
advantages: 

•  It  will  be  of  minimal  risk  to  the  personnel  performing  the  measurements. 

•  The  method  may  be  able  to  cover  large  tracts  of  land  in  a  relatively  short  time. 

•  The  assessments  possibly  may  be  done  at  a  reasonable  cost. 

•  It  could  be  used  to  locate  “potential  hot  regions”  for  buried  UXO  in  cases 
when  surface  debris  is  correlated  with  buried  UXO. 

The  technology  has  important  limitations,  however: 

•  REMIDS  relies  on  direct  line  of  sight  for  all  three  sensors.  It  would  be  risky  to 
rely  on  this  technology  in  areas  with  broad-leaf  vegetation  or  trees.  Further, 
any  buried  UXO  will  not  be  detected  by  REMIDS.  Even  a  thick  layer  of  dust 
could  compromise  a  sensor’s  reading,  leading  to  a  lower  detection  probability. 

•  Because  different  types  of  UXO  will  have  different  reflectance,  polarization, 
and  thermal  signatures,  a  priori  knowledge  of  the  ordnance  type  is  important 
to  calibrate  REMIDS.  Thus,  REMIDS  may  not  reliably  detect  unexpected  or 
uncharacterized  ordnance. 

•  REMIDS  may  not  reliably  detect  small  ordnance  when  the  background  has 
characteristics  similar  to  the  UXO.  In  tests  performed  at  the  Yuma  Proving 
Ground,  it  was  necessary  to  require  that  UXO  candidates  contain  a  minimum 
number  of  contiguous  pixels  to  minimize  the  background  contribution  to  the 
candidate  pool. 

•  Currently,  REMIDS  relies  heavily  on  the  operator  to  discriminate  UXO  from 
background  in  challenging  environments.  In  the  last  stage  of  the  Yuma  data 
analysis,  the  operator  determined  candidates  based  on  the  shape  of  objects  as 
viewed  by  a  particular  sensor  channel,  usually  polarization.  This  raises 
questions  concerning  operator  training  and  operator-to-operator  variability. 

C.  SYSTEM  PERFORMANCE 

The  REMIDS  system  was  tested  at  a  2,400  m^  test  site  at  Fort  Rucker,  Alabama, 
and  at  a  0.5  km^  test  site  at  Yuma  Proving  Ground  in  Arizona.  The  performance  of  this 
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system  is  site  dependent.  Some  sites  consist  of  backgrounds  that  render  discrimination 
based  solely  on  the  three  spectral  returns  impossible.  If  the  background  is  grassy,  it  is 
quite  likely  that  a  Pj  of  at  least  90  percent  is  achievable  with  very  low  false-alarm  rates 
using  only  the  spectral  information  (see  Figure  ES-1,  the  Fort  Rucker  performance 
curve). 
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Figure  ES-1.  Receiver-Operator  Characteristic  (ROC)  Curve  for  Fort  Rucker:  Aii  Targets 

On  the  other  hand,  at  sites  like  Yuma,  such  performance  is  probably  impossible 
with  just  the  spectral  information.  However,  with  a  knowledgeable  operator,  fair-to-good 
performance  can  be  expected  once  a  minimum  size  requirement  is  established  for 
potential  UXO  candidates.  Figure  ES-2  shows  the  probability  of  detection  as  a  function 
of  radius  surrounding  the  target  at  Yuma,  where  only  ordnance  items  greater  than  4  pixels 
in  size  were  included  in  the  target  baseline.  The  position  accuracy  of  the  REMIDS  system 
at  Yuma  was  determined  to  be  1.55  m  in  the  easting  direction  and  2.18  m  in  the  northing 
direction,  and  a  very  low  false-alarm  rate  of  41  per  km^  was  achieved.  We  note  that  the 
performance  indicated  in  Figure  ES-2  is  probably  a  lower-bound  estimate  for  the 
potential  of  this  system,  because  there  were  some  areas  of  the  site  that  the  helicopter  did 
not  fly  over,  and  these  “holiday  areas”  contained  an  unknown  number  of  targets.  On  the 
other  hand,  the  target  recognition  capabilities  of  the  operator  played  a  crucial  role  in  the 
performance  of  this  system  at  Yuma,  and  it  is  not  known  how  much  Pj  will  vary  with 
operator  experience  and  training. 
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Figure  ES-2.  REMIDS  Performance  at  Yuma 


— I — 1 — I — 1 — 

■  '  ‘  -n 

- 1 - i - ! - ! - 

► - 

— i — ; — 1 — 1 — 

,  . _ 

1  '  '  '  'n 
- - 

■  '  ^ 

- H 

. 

— 1 — 1 — 1  : 

► 

1 . 

y 

1 - ^ 

- 

- 

- 

- . m . %  Detected 

correctly  ID’d  J 

- 

/ 

/ 

/ 

- 

1 

_-„i ,.,l _ 1 — 1 — 

. 7 . 

/ 

1  .  . 

1 _ 1 — I — 1 — i — 

t  •  t  * 

i . . 

_ 1 _ 1 — 1 — i — 

# 


# 


ES-4 


I.  INTRODUCTION 


Millions  of  acres  of  government  land  are  contaminated  with  unexploded  ordnance 
(UXO),  a  result  of  years  of  testing  and  training  in  the  armed  forces.  As  part  of  the  effort 
to  prepare  some  of  this  land  for  use  other  than  as  test  ranges,  programs  are  underway  to 
develop  methods  to  safely  and  reliably  detect  UXO  so  that  the  contaminated  sites  may  be 
cleaned  before  realignment. 

This  report  describes  the  REmote  Minefield  Detection  System  (REMIDS)  devel¬ 
oped  by  the  U.S.  Army  Engineer  Waterways  Experiment  Station  to  detect  surface  UXO, 
and  reviews  the  performance  of  REMIDS  in  tests  at  the  Yuma  Proving  Ground,  Arizona, 
and  at  Fort  Rucker,  Alabama. 

The  principle  behind  REMIDS  is  to  enhance  the  discrimination  of  surface  UXO 
by  relying  on  multiple  signatures:  surface  UXO  may  exhibit  a  unique  combination  of 
reflectance,  polarization,  temperature,  and  footprint  (shape),  compared  to  natural  objects 
in  the  UXO’s  surroundings.  Discrimination  based  on  four  signatures  is  in  principle 
greater  than  that  based  on  fewer  signatures. 

The  REMIDS  hardware  consists  of  an  airborne  line  scanner  with  sensors  that 
measure  the  reflectance,  polarization,  and  thermal  response  in  710  round  “spots,”  each  of 
which  subtend  1.25  mrad.  (The  reflectance  and  polarization  sensors  are  active  and  utilize 
a  Nd:YAG  laser;  the  thermal  response  sensor  is  passive.)  The  710  spots  partially  overlap 
and  are  arranged  in  a  line  such  that  the  total  field  of  view  of  one  scan  line  is  1 .25  mrad  by 
40  deg.  The  scan  rate  of  the  device  is  350  lines  per  second.  Thus,  when  the  REMIDS  is 
flown  in  a  helicopter  at  an  altitude  of  130  ft  and  a  speed  of  32  knots,  each  of  the  three 
sensors  will  digitize  its  own  map  of  the  ground  level  with  a  pixel  size  of  1 .9  in.  x  1 .9  in. 

Analysis  of  the  REMIDS  digitized  data  is  performed  in  three  stages.  In  the  first 
stage,  a  computer  classifies  each  pixel  as  from  a  UXO  candidate  or  from  the  background. 
In  the  second  stage,  a  requirement  on  the  minimum  size  of  objects  (i.e.,  on  the  number  of 
contiguous  pixels)  is  imposed.  In  the  last  stage,  an  operator  views  all  potential  candidates 
and  determines  if  they  are  UXO  or  background.  The  operator  may  also  classify  the  type 
of  UXO. 


The  REMIDS  technology  has  both  advantages  and  limitations  in  locating  UXO. 

As  an  airborne  system  based  on  current  technology,  it  promises  three  advantages: 

•  It  will  be  of  minimal  risk  to  the  personnel  performing  the  measurements. 

•  The  method  may  be  able  to  cover  large  tracts  of  land  in  a  relatively  short  time. 

•  The  assessments  possibly  may  be  done  at  a  reasonable  cost. 

The  technology  has  important  limitations,  however: 

1 .  REMIDS  relies  on  direct  line  of  sight  for  all  three  sensors.  It  would  be  risky  to 
rely  on  this  technology  in  areas  with  broad-leaf  vegetation  or  trees.  Further, 
any  buried  UXO  will  not  be  detected  by  REMIDS.  Even  a  thick  layer  of  dust 
could  compromise  a  sensor’s  reading,  leading  to  a  lower  detection  probability. 

2.  Because  different  types  of  UXO  will  have  different  reflectance,  polarization, 
and  thermal  signatures,  a  priori  knowledge  of  ordnance  type  is  important  to 
calibrate  REMIDS.  Thus,  REMIDS  may  not  reliably  detect  unexpected  or 
uncharacterized  ordnance. 

3.  REMIDS  may  not  reliably  detect  small  ordnance  when  the  background  has 
characteristics  similar  to  the  UXO.  In  tests  performed  at  the  Yuma  Proving 
Ground,  it  was  necessary  to  require  that  UXO  candidates  contain  a  minimum 
number’of  contiguous  pixels  in  order  to  minimize  the  background  contribution 
to  the  candidate  pool  (this  is  discussed  below  in  the  performance  assessment 
section  of  this  report). 

4.  Currently,  REMIDS  relies  heavily  on  the  operator  to  discriminate  UXO  from 
background  in  challenging  environments.  In  the  last  stage  of  the  Yuma  data 
analysis,  the  operator  determined  candidates  based  on  the  shape  of  objects 
as  viewed  by  a  particular  sensor  channel,  usually  polarization.  This  raises 
questions  concerning  operator  training  and  operator-to-operator  variability. 
(Though  it  is  not  currently  part  of  the  data  reduction  flow,  it  may  be  possible 
to  automate  the  shape  discrimination  algorithm  in  future  implementations.) 

In  summary,  REMIDS  can  be  used  to  detect  known  types  of  surface  UXO  in  areas 
where  there  is  a  direct  line  of  sight  from  the  air  to  the  ground  and  good  distinction 
between  UXO  and  background  spectral  returns.  REMIDS  could  be  used  to  locate 
“potential  hot  regions”  for  buried  UXO  in  circumstanees  when  surface  debris  could  be 
correlated  with  buried  UXO.  For  example,  surface  debris  at  an  impact  point  of  a  bombing 
or  artillery  range  may  be  detected  by  REMIDS,  and  this  could  betray  possible  buried 
UXO.  In  this  situation,  follow-up  detection  methods  (such  as  magnetometers)  would  have 
to  be  employed  to  determine  the  existence  and  location  of  any  buried  UXO. 
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Section  11  of  this  report  provides  a  detailed  technical  description  of  the  REMIDS 
apparatus.  Section  III  is  an  assessment  of  the  system’s  performance  at  Yuma,  Arizona, 
and  at  Fort  Rucker,  Alabama.  Section  IV  gives  a  brief  cost  assessment. 


II.  TECHNICAL  DESCRIPTION 


A.  HARDWARE  DESCRIPTION 

The  REMIDS  system  consists  of  an  active/passive  line  scanner,  real-time  process¬ 
ing  and  display  equipment,  and  navigational  equipment.*  The  scanner  collects  three 
channels  of  optically  aligned  image  data  consisting  of  two  active  laser  channels  and  a 
passive  thermal  infrared  channel.  Figure  II- 1  shows  the  common  optics  for  the  system. 
Table  IM  gives  the  specifications  of  the  system,  with  information  presented  for  the  three 
spectral  channels.  The  “P  channel”  senses  the  reflected  light  polarized  parallel  to  the 
transmitted  light,  while  the  “C  channel”  senses  the  reflected  light  polarized  perpendicular 
to  the  transmitted  light.  The  reflectance  is  essentially  the  sum  of  the  two  channels,  while 
the  polarization  is  the  normalized  difference.  (If  a  beam  of  polarized  light  is  incident  on  a 
perfectly  smooth  surface,  the  reflected  beam  will  possess  the  same  polarization  as  the 
incident  beam,  and  in  this  limit,  the  return  in  the  C  channel  will  be  zero,  and  the 
polarization  return  will  be  maximum.) 


Table  IM.  REMIDS  Operational  Specifications 


Digitized  Field  of  View 

40  deg 

instantaneous  Field  of  View 

1.25  mrad 

Scan  Rate 

350  lines/sec 

Scan  Width  (at  an  altitude  of  130  ft) 

29  m 

Digitized  Samples  per  Scan  Line 

710  per  channel 

Roll  Correction 

±  15  deg 

Laser  Type 

Nd:YAG 

Laser  Frequency 

1 .064  pm 

Spectral  Channels 

P  Channel 

1.064  pm 

C  Channel 

1 .064  pm 

Thermal  Channel 

8-12  pm 

*  J.H.  Ballard,  R.M.  Castellane,  B.H.  Miles,  and  K.G.  Wesolowicz,  The  Remote  Minefield  Detection 
System  (REMIDS)  II  Major  Components  and  Operation,  Technical  Report  EL-92-30,  U.S.  Army 
Engineer  Waterways  Experiment  Station,  Vicksburg,  Mississippi,  1992. 


Using  the  specifications  given  in  Table  II- 1,  a  helicopter  flying  at  130  ft  will 
produce  an  instantaneous  pixel  size  of  1 .9  in.  x  1 .9  in.^  A  speed  of  32  knots  will  maintain 
the  aspect  of  the  image  being  constructed  as  the  data  are  collected. 

B.  PHENOMENOLOGY 

The  REMIDS  surface  mine/UXO  detection  algorithm  comprises  three  sequential 
steps.  In  the  first  step,  the  reflectance,  polarization,  and  thermal  returns  are  used  to  assign 
a  classification  to  each  pixel.  Such  a  classification  provides  information  on  material  type 
and  allows  for  some  reduction  in  the  total  number  of  pixels  that  must  be  investigated.  In 
the  second  step,  pixels  of  like  classification  are  joined  to  form  objects.  Each  object  is 
characterized  by  its  total  area,  boundary,  and  distance  between  its  two  farthest  pixels.  At 
the  end  of  the  second  step,  a  table  is  generated  that  groups  the  objects  according  to 
location  and  gives  both  the  spectral  and  size/shape  information  of  each  object.  In  the  third 
step,  the  operator  decides  which  objects  are  targets,  using  the  table  generated  in  the 
second  step  to  guide  his  decisions.  In  the  following  paragraphs,  we  discuss  each  of  these 
stages  in  some  detail,  and  we  provide  a  quantitative  estimate  of  the  role  of  each  step  in 
reducing  the  false  alarms. 

1.  Stage  1:  Spectral  Discrimination 

The  first  step  in  the  algorithm,  which  we  call  “Stage  1,”  relies  on  the  fact  that 
different  materials  will  yield  different  polarization,  reflectance,  and  thermal  responses. 
The  reflectance  is  a  function  of  both  the  refractive  index  of  the  object  and  its  orientation 
relative  to  the  incident  light.  In  general,  metals  will  have  a  very  high  reflectance  because 
they  have  a  complex  refractive  index,  which  essentially  means  that  they  do  not  permit  the 
penetration  of  an  electromagnetic  wave  to  any  significant  depth  and  thus  reflect  almost 
all  of  the  incident  light.  (In  the  limit  of  a  perfect  conductor,  the  penetration  depth  goes  to 
zero  and  the  reflectivity  goes  to  unity.)^ 

The  polarization  of  a  material  is  essentially  a  measure  of  its  “smoothness,”  which 
according  to  the  Rayleigh  criterion,  is  proportional  to  )Jih^,cosQ),  where  X  is  the  wave¬ 
length  of  the  incident  light,  0  is  the  angle  of  incidence,  and  is  the  RMS  height  of  the 
surface.  A  material  can  therefore  appear  “rough”  and  hence  yield  a  low  polarization 


2  The  distance  between  pixel  centers  is  1 .6  in.,  implying  an  overlap  between  pixels. 

3  To  be  more  precise,  the  reflectivity  of  a  metal  is  very  high  as  long  as  the  frequency  of  the  light  is 
below  the  plasma  frequency  of  the  metal.  Aluminum  will  reflect  light  at  all  optical  wavelengths. 


return  at  1.06  ^im,  and  yet  appear  “smooth”  and  yield  a  high  polarization  return  at 
10.6  |xm.  Hence,  one  cannot  predict  the  polarization  return  of  a  material  based  on  how  it 
looks  or  feels.  One  example  of  this  is  rust,  which  one  might  intuitively  expect  to  have  low 
polarization,  but  which  has  been  found  to  yield  a  significant  polarization  return  at 

1 .06  |i.m. 

The  thermal  returns  of  both  ordnance  and  background  will  depend  upon  their 
thermal  properties  as  well  as  on  the  time  of  day  and  the  weather  conditions.  The  discrimi¬ 
nation  capabilities  of  the  thermal  channel  should  be  best  soon  after  sunrise  and  sunset, 
when  objects  with  different  thermal  diffusivities,  surface  absorptivities,  and  emissivities 
will  heat  up  and  cool  down  at  different  rates.  Near  midday,  when  thermal  equilibrium  is 
approached,  the  thermal  discrimination  between  objects  will  be  more  difficult.^ 

The  classification  of  the  test  site  pixels  according  to  their  spectral  information  is  a 
complex  process.  For  each  pixel  in  question,  a  set  of  distances,  D„  is  defined  for  each  of 
the  possible  target  material  classifications: 

,  (1) 

lit  J  t  J  I  J 

where  i  is  the  ith  classification,  angular  brackets  indicate  the  mean,  and  a  denotes 
standard  deviations.  The  means  and  standard  deviations  are  obtained  from  the  pixels  that 
come  from  the  targets  in  the  calibration  site,  and  the  classifications  include  only  potential 
target  materials.  The  parameters  Wpi,  w*,  and  for  the  polarization,  reflectance,  and 
thermal  channels,  respectively,  are  weights  that  account  for  the  fact  that  the  pixels  in  the 
calibration  site  are  not  necessarily  representative  of  those  in  the  test  site.  For  example, 
due  to  time  drift,  there  should  be  much  more  variability  in  the  thermal  channel  for  the  test 
site  than  for  the  calibration  site.  Furthermore,  to  reduce  the  complexity,  only  a  few 
classifications  are  chosen.  Thus,  several  different  materials  will  fall  into  one  general 
class,  implying  greater  variability  in  all  three  channels  for  the  test  site  than  for  the 
calibration  site.  Typically,  at  Yuma,  the  values  of  w^.  were  chosen  to  be  close  to  unity 
because  the  values  of  Op,  were  large;  those  of  w^,  were  around  two;  and  those  of  were 

at  least  two,  if  not  greater. 

For  each  test  site  pixel,  the  smallest  D„  determines  its  target  material 

classification  if  is  less  than  or  equal  to  a  threshold  value,  I>,h„sh-  If  ^Lmm  exceeds 

4  See  performance  section  for  the  Fort  Rucker  data,  Section  III.A,  where  it  is  shown  that  the  discrimi¬ 
nation  capabilities  of  the  thermal  channel  at  07:3 1  a.m.  are  far  superior  to  those  at  1 1 :22  a.m. 


Djhresh.  then  the  pixel  is  classified  as  background.  is  an  adjustable  parameter,  but  it  is 
assumed  to  be  the  same  value  for  each  material  classification.  On  the  other  hand,  the 
quantities  Wpi,  and  may  vary  from  one  classification  to  another.  Thus,  the  total 
number  of  adjustable  parameters  is  3N  +  1,  where  N  is  the  number  of  target  material 
classifications.  We  note  that  by  fixing  the  3N  values  of  Wpi,  Wm,  and  and  varying  only 
^thresh*  on®  onn  build  up  a  receiver  operating  characteristic  (ROC)  curve. 

The  3N+ I  adjustable  parameters  are  first  chosen  to  optimize  the  results  from  the 
calibration  site.  Optimizing  the  results  is  a  nontrivial  task  itself  and  can  take  as  much  as 
one  work  day  to  complete.  If  something  does  not  seem  right  when  Eq.  (1)  is  evaluated  for 
the  pixels  of  the  test  site  (e.g.,  that  pixels  covering  a  very  large  area  are  all  classified  as 
being  the  same  target  material),  the  operator  will  vary  some  of  the  3A^  +  1  values  and 
repeat  the  process.  Thus,  the  calibration  site  provides  the  operator  with  initial  values  for 
each  of  the  adjustable  parameters,  but  several  iterations  through  the  test  site  data  are 
usually  necessary  to  obtain  the  optimal  set  of  parameters. 

Table  II-2  provides  the  means  and  standard  deviations  of  P,  R,  and  T  obtained 
from  a  noontime  flight  at  Yuma  Proving  Ground  on  June  26,  1996.  The  first  column 
gives  the  number  of  pixels  used  to  generate  the  statistics  for  each  material.  Those 
materials  marked  with  asterisks  are  the  target  material  classifications  used  in  the  evalu¬ 
ation  of  the  test  site  data.  The  means  and  standard  deviations  of  all  except  the  dielectric 
mines  were  obtained  from  the  calibration  site;  the  means  and  standard  deviations  of  the 
dielectric  mines  were  taken  from  mines  in  the  test  site  because  none  were  present  at  the 
calibration  site.^ 

In  Table  II-2  it  may  be  surprising  that  the  reflectivity  of  aluminum  is  lower  than 
that  of  many  other  entries.  This  occurs  because  the  aluminum  targets  used  to  obtain  these 
numbers  in  the  calibration  site  were  very  small.  Thus,  edge  pixels,  which  were  affected 
by  the  background,  had  a  significant  effect  on  the  mean.  Conversely,  large  sheets  of 
aluminum  yield  strong  polarization  and  reflectance  returns. 

Two  other  entries  warrant  further  clarification.  “Desert  pavement”  refers  to  a 
rocky,  blackened  crust  that  covers  much  of  the  test  site  at  Yuma  and  which  appears  quite 


^  One  might  wonder  how  they  were  able  to  classify  objects  as  dielectric  mines,  given  that  there  were 
none  in  the  calibration  site.  It  turns  out  that  it  was  known  that  one  dielectric  mine  was  located  at  the 
comer  of  the  test  site.  This  one  mine  provided  about  25-30  pixels  of  information  on  P,R,  and  T. 
Although  the  standard  deviation  on  25-30  pixels  is  large,  this  provided  a  starting  point  for  the  analysis 
of  subsequent  objects.  A  few  additional  objects  with  similar  returns  were  then  picked  up  and  added  to 
the  database  for  dielectric  mines. 


smooth  at  1.06  ^im,  yielding  fairly  high  polarization  returns.  “Desert  varnish,”  a  back¬ 
ground  feature  yielding  even  higher  polarization  returns,  refers  to  the  rocks  covered  with 
a  glass-like  sheen.  These  rocks  were  located  mainly  in  the  wash  areas  of  the  test  site.^  At 
some  sites,  such  as  Fort  Rucker,  the  polarization  channel  alone  can  provide  very  good 
distinction  between  the  targets  and  the  background,  but  at  Yuma  this  is  clearly  not  the 
case.  The  additional  information  provided  by  the  thermal  channel  does  allow  for  desert 
pavement  to  be  fairly  easily  separated  from  potential  targets,  but  even  with  the  informa¬ 
tion  from  all  three  channels,  desert  varnish  is  very  difficult  to  distinguish  from  the  iron/ 
olive  drab  paint  class. 


Table  11-2.  Polarization,  Reflectance,  and  Thermal  Returns  at  Yuma 


Material 

#  Pixels 

<p> 

ap 

<R> 

<T> 

-T 

desert  pavement 

21,000 

149.91 

9.83 

62.27 

9.24 

148.82 

15.14 

sand  and  fines 

2523 

105.17 

7.18 

166.00 

7.97 

129.11 

5.45 

desert  varnish 

189 

180.57 

15.27 

92.58 

12.19 

108.93 

13.66 

iron  (oxidized) 
and  olive  drab 
paint  * 

192 

206.98 

35.22 

58.72 

22.43 

98.96 

17.62 

white  paint* 

15 

197.07 

36.69 

145.67 

22.60 

73.20 

17.50 

aluminum* 

80 

223.30 

31.98 

101.85 

27.95 

76.48 

33.53 

dielectric  mines 
(white  piastic)* 

180 

49.58 

20.72 

131.66 

13.60 

90.67 

15.93 

We  note  that  completion  of  Stage  1  of  the  REMIDS  algorithm  does  not  require 
the  helicopter  to  obtain  the  calibration  data,  land,  process  the  data,  and  then  obtain  the 
target  data.  All  polarization,  reflectance,  and  thermal  data  are  post  processed.  Thus,  the 
REMIDS  approach  is  to  fly  over  the  calibration  site  and  then  fly  directly  to  the  real  site. 
In  fact,  this  is  necessary  for  thermal  channel  calibration  data  to  have  any  applicability. 

The  completion  of  Stage  1  results  in  a  significant  reduction  of  the  total  number  of 
pixels  that  might  contain  targets  of  interest.  Nonetheless,  Stage  1  cannot  always  be  relied 
upon  to  provide  sufficient  discrimination  from  the  background  to  be  a  viable 
surface/UXO  mine  detection  system  in  and  of  itself.  This  is  particularly  true  at  sites  such 
as  Yuma,  where  the  existence  of  desert  varnish  resulted  in  many  tagged  pixels.  Other 


6  A  wash  area  is  an  area  where  flash  floods  caused  a  lot  of  water  of  flow  through  the  area.  These  areas 
represented  roughly  30  percent  of  the  site. 


sites  may  fare  better;  for  example,  as  shown  in  Table  II-2,  sand  is  easily  distinguishable 
from  the  targets. 

In  addition  to  being  site  dependent,  the  reduction  in  total  pixels  of  interest  after 
Stage  1  is  also  dependent  on  the  choice  of  D,hresh*  ^  larger  D,hresh  means  that  more  pixels 
will  be  tagged  as  possible  targets.  The  optimal  value  of  is  typically  determined  after 
a  few  iterations  through  the  entire  algorithm.  Usually,  the  operator  will  begin  with  a 
conservative  value  of  D^i,  and  focus  on  a  few  difficult-to-distinguish  items  that  he 
believes  are  targets.  He  will  then  decrease  i^thresh  uittil  those  objects  are  no  longer  cued.  At 
Yuma,  a  conservative  value  of  =  1.6  resulted  in  a  reduction  in  the  number  of  tagged 
pixels  after  Stage  1  by  about  2-1/2  orders  of  magnitude,  from  about  300  million  pixels 
(the  total  overflown  area)  to  about  600,000. 

2.  Stage  2:  Size/Shape  Discrimination 

In  the  second  step  of  the  REMIDS  algorithm,  which  we  call  Stage  2,  pixels  of 
like  material  classification  are  joined  to  form  objects.  Figure  II-2  illustrates  an  example  in 
which  10  pixels  of  material  class  1  are  combined  to  form  an  object  with  an  area  of 
36.1  in^;  a  perimeter  of  30.4  in.;  and  a  length,  defined  as  the  distance  between  the  two 
furthest  pixels,  of  10.75  in.  (each  pixel  is  1.9  in.  x  1.9  in.). 
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Figure  11-2.  Joining  Objects  of  Like  Classification 

Software  has  been  developed  which  performs  this  step  automatically.  The 
operator  has  the  option  of  specifying  how  he  wants  the  objects  to  be  formed;  for  example, 
he  may  specify  that  pixels  must  be  contiguous,  or  that  they  need  not  be.  Furthermore,  the 
operator  can  customize  the  software  so  that,  in  addition  to  forming  objects  from  tagged 
pixels,  it  also  filters  out  many  of  the  objects  based  on  their  size  and/or  shape.  For 
example,  an  object  with  a  very  large  area  may  be  rejected,  as  may  one  that  is  too  small, 
too  thin,  or  irregularly  shaped. 

The  level  of  discrimination  provided  by  the  size/shape  filter  greatly  affects  the 
number  of  objects  that  are  passed  on  to  the  operator:  too  little  pre-screening  in  Stage  2 


will  result  in  too  many  man-hours  spent  in  the  final  step,  but  too  much  pre-screening  will 
result  in  too  many  missed  targets.  Just  as  with  the  3N  +  1  spectral  parameters  in  Stage  1, 
the  optimal  level  of  discrimination  used  in  Stage  2  is  ultimately  determined  after  several 
iterations  through  the  data.  At  Yuma,  for  example,  it  was  decided  to  set  the  minimum  size 
such  that  any  object  smaller  than  four  pixels  was  rejected.  Doing  so  greatly  reduced  the 
number  of  false  alarms  due  to  desert  varnish,  but  at  the  expense  of  missing  targets  smaller 
than  four  pixels,  such  as  grenades  and  Valmeira  mines.  On  the  other  hand,  discrimination 
based  on  “irregular  shape”  was  found  to  be  unreliable  at  Yuma  because  variabilities  in 
helicopter  air  speed  caused  oblong  targets  to  appear  to  be  irregularly  shaped.  Thus,  the 
only  discrimination  provided  by  the  size/shape  filter  in  Stage  2  for  the  Yuma  data  was 
that  of  the  minimum  size  requirement. 

The  output  of  the  size/shape  filter  in  Stage  2  is  a  table  in  which  objects  are 
ordered  by  location  (image  and  scan  line).  For  each  object,  the  material  classification  and 
shape  information  (area,  boundary,  length)  are  given.  At  Yuma,  for  the  iteration  in  which 
^thresh  was  set  to  1.6,  where  600,000  pixels  were  passed  to  the  size/shape  filter,  approxi¬ 
mately  20,250  candidate  objects  were  selected.  Of  these,  roughly  20,000  were  not  targets. 

3.  Stage  3:  Operator  Discrimination 

The  table  provided  to  the  operator  by  the  shape/size  filter  allows  him  to  decide  on 
which  images  to  focus  his  attention.  The  operator  chooses  a  spectral  channel  (usually  the 
polarization  channel)  and  scrolls  through  the  images,  viewing  several  suspicious  objects 
at  once.  Usually,  the  man-made  ordnance  items  are  easily  discernible  by  eye.^  Thus,  a 
trained  operator  is  able  to  identify  many  of  the  ordnance  items  and  eliminate  many  of  the 
false  alarms  very  quickly,  without  having  to  study  each  individual  object  in  great  detail. 
Those  objects  that  the  operator  cannot  identify  or  eliminate  right  away  are  carefully 
examined  in  each  of  the  three  spectral  channels.  We  note  that  the  operator  not  only  deter¬ 
mines  whether  a  suspicious  object  is  a  target,  he  also  classifies  the  target  if  he  decides 
that  it  is  an  ordnance  item. 

For  example,  at  Yuma,  about  95  percent  of  the  roughly  20,000  false  alarms  were 
located  in  the  wash  areas,  which  is  where  most  of  the  desert  varnish  was  located.  After 
carefully  examining  a  few  of  these  objects,  the  operator  realized  that  most  of  the  items 


7  Although  man-made  ordnance  is  fairly  easily  distinguished  from  natural  backgrounds,  it  should  be 
noted  that  it  may  not  be  easily  separated  from  man-made  clutter.  The  test  site  at  Yuma  was  a  fairly 
“clean”  site,  with  very  little  anthropic  clutter. 


located  in  these  areas  were  false  alarms.  Thus,  the  operator  was  able  to  scroll  through 
about  19,000  objects  fairly  quickly,  without  having  to  focus  on  any  of  them  in  detail.^ 

It  is  clear  that  the  role  of  the  operator  is  crucial  to  the  success  of  this  system.  The 
operator  is  involved  in  each  of  the  three  stages,  and  the  total  time  involved  is  dictated  by 
his  efficacy  and  skill.  For  the  data  collected  at  Yuma,  1  day  was  devoted  to  processing 
the  calibration  site  data  and  choosing  the  initial  values  of  the  3A^  +  1  spectral  parameters. 
The  analysis  of  the  test  site  data  then  took  3  days,  the  vast  majority  of  which  was  due  to 
time  spent  by  the  operator.  Of  those  3  days,  roughly  60  percent  of  that  time  was  spent 
scrolling  through  images,  while  the  remainder  of  the  time  was  dedicated  to  the  detailed 
examination  of  the  roughly  100  suspicious  objects  that  could  not  be  quickly  identified. 

Given  that  an  area  of  only  0.5  km^  entailed  3  man-days  of  tedious  data  analysis, 
an  obvious  question  to  address  is  whether  the  role  of  the  operator  can  be  automated. 
Clearly,  it  should  be  possible  to  reduce  the  amount  of  time  spent  scrolling  through 
images,  since  that  time  is  deteimined  primarily  by  the  level  of  pre-screening  provided  by 
the  size/shape  filter.  At  Yuma,  this  pre-screening  was  limited  to  just  a  minimum  size 
requirement.  The  developers  of  RJEMIDS  believe  that  a  robust  size/shape  filter  to  allow 
extensive  filtering  of  the  objects  before  they  are  passed  to  the  operator  can  be  developed, 
but  that  idea  has  yet  to  be  tested.  However,  if  one  opts  to  rely  heavily  on  a  size/shape 
filter  to  screen  the  objects,  then  one  should  expect  some  degradation  in  performance  over 
that  of  the  current  system,  because  it  is  unlikely  that  any  computer  can  provide  the  level 
of  discrimination  of  the  human  eye.  For  example,  the  wash  areas  at  Yuma  contain  many 
irregularly  shaped,  vamish-covered  rocks.  These  irregularities,  although  perceptible  to 
the  human  eye,  would  not  be  so  easily  distinguished  by  a  computer-based  algorithm.  In 
addition,  as  the  operator  scans  the  images,  he  develops  an  “intuition”  for  where  the 
targets  and  false  alarms  are  most  likely  to  be  located.  That  is,  humans  possess  pattern 
recognition  capabilities  that  may  be  difficult  to  replicate  in  a  computer  program. 
Nonetheless,  it  seems  impractical  to  rely  on  the  operator  to  perform  the  bulk  of  the 
discrimination  for  large  areas. 


8  There  were  some  ordnance  items  located  in  the  wash  area  as  well.  Some  of  these  targets  were 
discernible  based  on  their  size,  while  others  could  be  distinguished  by  their  spectral  signature.  Further¬ 
more,  compared  with  the  man-made  targets,  the  rocks  were  often  irregularly  shaped.  However,  some 
targets  were  lost;  small  items  made  of  iron  or  covered  in  olive  drab  paint  would  have  been  particularly 
difficult  to  pick  out. 


II-9 


in.  PERFORMANCE  ASSESSMENT 


In  this  section,  we  summarize  the  performance  of  the  REMIDS  system  at  two  test 
sites;  Fort  Rucker,  Alabama,  and  Yuma,  Arizona.  The  performance  at  Fort  Rucker  was 
superior  because  the  background  was  mainly  grass.  In  essence.  Fort  Rucker  looked  like  a 
“short  rough”  on  which  the  targets  were  easily  seen,  even  with  just  the  polarization 
information.  At  Yuma,  the  existence  of  desert  varnish,  with  its  high  polarization  return, 
made  the  targets  much  more  difficult  to  distinguish,  and  a  4-pixel  minimum  object  size 
was  required  in  the  analysis.  This  resulted  in  a  limitation  of  the  size  of  the  objects  that 
could  be  detected  at  Yuma  (e.g.,  grenades  fall  below  this  size  limit). 

A.  FORT  RUCKER  RESULTS 

Figure  III- 1(a)  shows  the  performance  of  the  individual  spectral  channels  with 
respect  to  the  detection  of  aluminum  items,  and  Figure  III- 1(b)  shows  the  performance  of 
the  individual  spectral  channels  with  respect  to  ferrous  and  painted  surface  items. 
Figure  III- 1(c)  shows  the  performance  of  the  combined  spectral  channels  for  the  detection 
of  both  aluminum  and  ferrous/painted  targets.  Figure  III- 1(a)  shows  that  the  polarization 
channel  alone  was  sufficient  to  detect  the  aluminum  ordnance  items:  100-percent  was 
achieved  with  only  15  false  alarms  in  2,400  m^  In  Figure  Ill-l(b),  the  performance  of  the 
polarization  channel  in  detecting  ferrous  or  painted  objects,  while  not  as  remarkable  as 
for  aluminum,  was  still  quite  good,  with  a  P ^  of  about  95  percent  at  15  false  alarms.  It  is 
clear  from  Figure  III- 1(c)  that  the  combination  of  the  three  spectral  channels  yielded 
excellent  performance:  greater  than  80-percent  P^  was  achieved  with  essentially  0  false 
alarms,  while  100-percent  P^  was  possible  with  just  20  false  alarms.  Thus,  the  spectral 
filter  alone  was  sufficient  at  Fort  Rucker;  no  size/shape  filter  was  required,  and  the 
operator’s  role  was  minimized. 

B.  YUMA  RESULTS 

The  performance  at  Yuma  is  somewhat  difficult  to  assess  quantitatively  for  two 
reasons.  First,  insufficient  information  was  provided  about  target  locations.  Documen¬ 
tation  consisted  solely  of  the  contents  of  each  25  m  x  25  m  cell,  with  a  rough  estimate  of 
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Figure  lll-1{a).  ROC  Curve  for  Fort  Rucker:  Aluminum  Targets  Oniy 
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Figure  lll-l(b).  ROC  Curve  for  Fort  Rucker:  Ferrous  and  Painted  Targets  Only 
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Figure  lll-l(c).  ROC  Curve  for  Fort  Rucker:  All  Targets 

the  target  location(s)  within  each  cell,  such  as  “center”  or  “northeast  comer.”  For  some 
cells,  a  hand-drawn  sketch  was  provided.  Clearly,  the  location  accuracy  of  the  REMIDS 
system  could  not  be  tested,  and  detections  could  not  be  correlated  to  targets  in  cells 
containing  multiple  items. 

The  second,  and  much  more  unsettling,  reason  for  the  difficulties  in  evaluating  the 
REMIDS  performance  at  Yuma  is  that  there  were  uncertainties  in  the  baseline.  After  the 
test  was  conducted,  the  REMIDS  team  walked  much  of  the  site,  taking  notes  on  target 
types  and  locations  within  each  cell.  They  found  that  for  some  of  the  cells,  there  seemed 
to  be  significant  discrepancies  between  what  was  documented  as  the  baseline  and  what 
they  actually  observed.  It  was  learned  that  the  site  is  somewhat  “dynamic”;  that  is,  items 
are  “borrowed”  and  moved  around.  The  documentation  provided  at  that  time  was  based 
on  a  November  1994  test  site  survey.  The  site  manager  agreed  to  provide  an  update  of  the 
site  baseline,  which  we  received  in  March  1997.  The  problem  is  that  it  is  not  known  for 
certain  how  much  the  site  changed  between  June  1996,  when  the  test  data  was  taken,  and 
March  1997,  when  the  update  to  the  baseline  was  determined.  Comparing  the  November 
1994  baseline  to  the  March  1997  baseline,  we  found  that  of  the  132  cells  that  actually 
contain  some  type  of  ordnance,  20  showed  significant  differences  between  the  two  time 
periods.  It  is  not  known  for  certain  whether  the  contents  of  all  20  cells  had  actually 
changed,  or  whether  the  documentation  for  some  of  those  cells  was  incorrect.  Either  way, 
however,  those  20  cells  must  be  considered  questionable. 
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The  REMIDS  team  attempted  to  grade  themselves  as  best  they  could,  given  these 
two  limitations.  However,  because  they  had  walked  much  of  the  site  and  had  been 
provided  with  the  November  1994  and  March  1997  baselines,  it  was  clearly  impossible 
for  them  to  grade  themselves  “blindly.”  Thus,  we  have  attempted  to  grade  the  REMIDS 
system  as  objectively  as  possible.  We  describe  our  methodology  below. 

First,  in  consult  with  our  sponsor,  we  decided  to  limit  our  focus  to  those  targets 
(1)  that  are  larger  than  the  4-pixel  threshold,  and  (2)  that  are  of  interest  to  the  UXO 
community.  The  targets  chosen  were  500-lb  bombs,  2.75-in.  rockets,  81 -mm  mortars, 
105-mm  projectiles,  and  155-mm  projectiles.  Dielectric  mines,  although  not  of  interest  to 
the  UXO  application  that  was  being  examined,  were  also  included  because  they  were  a 
particularly  easy  target  for  REMIDS  to  detect, ^  and  thus  provided  a  means  of  estimating 
the  location  accuracy  of  the  system.  Excluded  items  included  grenades,  Valmeira  mines, 
gator  mines,  volcano  mines,  and  painted  mines.  All  but  the  last  of  these  were  too  small  to 
be  detected  with  the  4-pixel  threshold. 

For  each  category  of  interest,  we  determined  in  which  cells  the  November  1994 
and  March  1997  documentation  was  consistent  and  clear.  In  October  1997,  we  submitted 
these  cells  (59  cells  altogether,  containing  186  targets)  to  the  site  manager  and  requested 
that  he  obtain  GPS  data  for  each  item  in  each  cell. 

Upon  receiving  the  requested  GPS  data  in  November  1997,  however,  we  dis¬ 
covered  that  the  number  of  items  in  6  of  the  59  cells  differed  from  what  was  documented 
in  March  1997.  We  had  chosen  those  59  cells  specifically  because  they  appeared 
according  to  the  1994  and  1997  documentation  to  remain  constant  over  time.  That  the 
contents  of  some  of  the  cells  seemed  to  have  changed  in  less  than  a  year  caused  us  to 
question  whether  the  GPS  coordinates  provided  for  the  remaining  53  cells  were  relevant 
to  the  June  1996  test.  (Certainly,  if  the  items  in  some  cells  had  changed,  the  items  in 
others  may  have  been  moved  around.) 

In  February  1998  the  site  manager  attempted  to  determine  what  had  happened  in 
those  six  cells.  He  concluded  that  while  two  of  the  cells  seemed  to  have  acquired  addi¬ 
tional  items,  for  the  other  four  cells  the  original  documentation  was  correct— the 
surveyors  simply  had  missed  some  items.  We  decided  to  discard  only  the  two  cells  where 
the  number  of  items  had  seemingly  increased.  In  so  doing,  we  may  slightly  overestimate 


9  See  Table  11-2.  The  dielectric  mines  gave  a  polarization  return  that  easily  distinguished  them  from  all 
other  targets  and  backgrounds. 


the  probability  of  detection,  but  we  estimate  that  the  effect  of  this  error  will  be  less  than 
1  percent.  If  we  assume  that  the  GPS  coordinates  of  the  items  in  the  remaining  57  cells 
have  not  changed  since  June  1996,  then  the  data  set  is  reduced  to  a  total  of  168  targets. 
However,  one  additional  cell  was  not  used  because  inadequate  information  was  provided 
by  the  REMIDS  team.^^  Thus,  the  useful  data  set  was  reduced  to  167  targets  in  56  cells. 

To  further  complicate  matters,  the  REMIDS  system  experienced  problems  with  its 
GPS  system  for  13  of  the  59  cells.  When  the  cells  in  which  no  GPS  data  were  collected 
by  REMIDS  and  the  cells  discussed  above  are  excluded,  a  total  of  141  targets  remain  in 
the  baseline  (43  cells). 

To  estimate  the  location  accuracy  of  the  REMIDS  system,  we  used  the  GPS  data 
for  easy  targets:  the  dielectric  mines  and  the  500-lb  bombs.  Seventeen  mines  and  two 
bombs  were  part  of  the  final  target  baseline;  however,  the  REMIDS  system  detected  only 
15  of  the  17  mines.  On  the  other  hand,  because  the  REMIDS  system  often  overflew  the 
same  target  twiee,  for  several  of  the  17  detected  mines  and  bombs,  there  were  two  sets  of 
GPS  coordinates.  In  sum,  27  data  points  were  used  to  calculate  location  accuracy. 

We  determined  the  location  accuracy  of  the  REMIDS  system  to  be  1.55  m  in  the 
easting  direction  and  2.18  m  in  the  northing  direction.  (The  precision  of  the  GPS  system 
used  at  the  site  was  0.1  m,  according  to  the  site  manager.)  These  values  lend  support  to 
the  assumption  that  the  GPS  coordinates  in  the  cells  whose  contents  presumably  have  not 
changed  are  relevant  to  our  analysis.  In  addition,  we  found  that  there  was  an  offset  bias  in 
the  REMIDS  position  compared  to  ground  truth:  1.12  m  in  the  easting  direction  and 
0.44  m  in  the  northing  direction.  We  left  these  offsets  in  the  data  during  our  analysis  of 
the  probability  of  detection  because  this  was  the  data  derived  from  the  test.  However,  we 
found  that  correcting  for  these  offsets  only  marginally  changes  the  device’s  detection 
efficiency. 


10  For  example,  say  a  cell  had  six  items  in  it,  but  only  five  GPS  coordinates  were  provided.  By  grading 
this  cell,  we  are  theoretically  giving  the  REMIDS  team  six  chances  at  five  objects.  However,  the  real 
effect  will  be  minimal  if  we  assume  that  the  items  are  randomly  placed  in  the  25  m  x  25  m  cell. 

1 1  The  cell  contained  one  dielectric  mine  and  two  painted  mines  on  top  of  a  4-ft  X  4-ft  aluminum  plate. 
The  REMIDS  team  noted  only  that  there  were  three  objects  on  top  of  a  plate,  and  gave  only  the  GPS 
coordinates  of  the  center  of  the  plate.  Because  the  dielectric  mines  were  used  only  to  determine  the 
location  accuracy  of  the  REMIDS  system,  we  decided  that  this  cell  was  not  needed. 

12  One  of  those  13  cells  is  also  one  of  the  4  cells  where  the  March  1997  documentation  indicated  more 
items  than  what  was  surveyed  in  November  1997.  Hence,  only  three  cells  having  fewer  surveyed  items 
than  what  was  documented  were  in  the  final  target  baseline. 


III-5 


We  then  estimated  the  probability  of  detection  as  a  function  of  radius  surrounding 
the  target.  To  perform  this  calculation  we  excluded  the  dielectric  mines,  which  are  not 
relevant  to  UXO  clearance.  Thus,  the  set  of  ground  truth  target  data  against  which 
REMIDS  was  graded  consisted  of  124  ordnance  items  (seventy  81 -mm  mortars,  twenty- 
four  105-mm  shells,  fifteen  155-mm  shells,  thirteen  2.75-in.  rockets,  and  two  500-lb 
bombs),  distributed  among  26  cells.  In  the  remaining  discussion,  “targets”  refer  to  ground 
truth  ordnance  items  emplaced  at  Yuma,  while  “candidates”  refer  to  REMIDS  detections. 
The  analysis  flow  was  structured  as  follows; 

1 .  Any  REMIDS  candidate  that  was  both  the  closest  candidate  to  a  target  and 
also  within  a  critical  radius,  Ren,,  of  the  target  was  matched  as  a  possible 
detection  of  the  target. 

2.  A  candidate  passing  the  requirement  in  (1)  was  considered  a  detection  of  a 
target  if  it  was  closer  to  that  target  than  to  all  other  targets.  The  matched 
candidate  and  target  were  then  removed  from  their  respective  lists. 

3.  Steps  (1)  and  (2)  were  repeated  for  the  reduced  lists  of  candidates  and  targets 
until  there  were  no  more  targets  or  there  were  no  more  candidates  within 

of  the  remaining  targets. 

4.  The  detection  probability  of  REMIDS  as  a  function  of  Ren,  was  computed  as 
the  number  of  detected  targets  for  a  given  divided  by  the  total  number  of 
targets  (124  in  this  analysis). 

5.  In  addition  to  the  detection  probability,  ordnance  identification  could  also  be 
tested  in  this  analysis  (the  REMIDS  team  used  imagery  from  the  polarization 
channel).  The  fraction  of  the  detected  ordnance  that  was  correctly  identified 
was  measured  as  a  function  of 

Table  III-l  and  Figure  III-2  summarize  the  results  of  this  analysis.  Table  III-l 
gives  the  breakdown  of  detection  probability  versus  for  each  ordnance  type. 
Figure  III-2  shows  the  probability  of  detection  and  probability  of  correct  identification  as 
a  function  of  for  the  combined  set  of  ordnance  items.  Using  Figure  III-2  and  taking 
an  example  value  of  =  5  m  (which  is  more  than  a  2  sigma  cut,  according  to  our 
estimate  of  the  system’s  position  resolution),  we  find  that  REMIDS  detected  76  percent 
of  the  target  test  sample  and  correctly  identified  95  percent  of  the  ordnance  detected. 

As  discussed  above,  there  was  an  offset  bias  in  the  REMIDS  position  measure¬ 
ment.  To  determine  how  an  offset  correction  would  affect  the  detection  probability, 
we  ran  a  Monte  Carlo  that  simulated  the  device’s  easting  and  northing  position  resolu¬ 
tions,  both  with  the  offsets  and  with  a  correction  that  removed  the  offsets.  For  the  case 


Table  111-1.  Probability  of  Detection  by  Ordnance  Type 
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Figure  III-2.  REMIDS  Performance  at  Yuma 

=  5  m,  we  found  that  correcting  for  the  offset  would  improve  the  detection  efficiency 
by  only  2.5  percent  over  the  uncorrected  sample.  Thus,  REMIDS  would  have  achieved 
about  a  78-percent  detection  probability  at  R^ri,  =  5  m  had  the  operators  corrected  the 
offset  bias  at  the  calibration  site.*^ 

We  note  that  our  calculation  of  Pj  does  not  account  for  holiday  areas,  which  are 
those  areas  not  overflown  by  the  helicopter  (see  Section  III.C  below).  Thus,  our  value  is 


The  effect  of  the  offset  bias  clearly  increases  as  decreases;  for  example,  at  -3  m,  the  offset 
bias  correction  would  yield  a  of  56  percent,  an  1 1 -percent  improvement  over  the  uncorrected  value 
of  51  percent  shown  in  Figure  III-2. 


likely  to  be  a  lower  bound  estimate  of  the  capability  of  this  system.  From  the  raw  data 
provided  to  us,  we  had  no  way  of  determining  which  targets  were  not  overflown  by  the 
REMIDS  system,  but  it  is  likely  that  the  effect  on  is  only  a  few  percent, 

To  estimate  the  false-alarm  rate  {FAR),  we  looked  at  three  regions  of  the  site  that 
contained  no  targets.  The  area  of  the  first  region  was  0.073  km^,  and  the  REMIDS  system 
declared  a  total  of  three  targets  in  that  area,  identifying  all  three  as  gator  mines.  This 
translates  into  a  FAR  of  41  per  km^.  The  second  region  was  0.049  km^  in  area,  and  the 
REMIDS  system  declared  two  targets,  one  a  gator  mine,  and  the  other  UXO  scrap.  This 
also  yields  a  FAR  of  41  per  km^.  The  third  region  was  a  much  smaller  area  of  only 
0.028  km2,  and  in  this  region,  the  REMIDS  system  declared  no  targets.  We  conclude, 
therefore,  that  the  REMIDS  system,  with  its  three-stage  analysis,  yields  a  very  low  false 
alarm  rate.  We  emphasize,  however,  that  the  false  alarm  rate  would  have  been  much 
higher  if  only  the  first  two  analysis  stages  had  been  employed  (spectral  and  size/shape), 
because  the  operator  played  a  crucial  role  in  recognizing  and  eliminating  false  alarms. 

In  addition  to  its  potential  use  in  identifying  individual  UXO  targets,  REMIDS 
also  has  potential  as  a  large-area  search  system  used  to  determine  which  sections  of  a 
large  area  must  be  thoroughly  investigated,  which  sections  will  require  only  minimal 
investigation  at  isolated  spots,  and  which  sections  can  be  declared  ordnance  free. 

To  illustrate  this  point.  Table  III-2  shows  the  number  of  false  alarms  per  square 
kilometer  after  stages  1  and  2  were  completed  for  a  number  of  different  ordnance-free 
sites  at  Yuma  and  Fort  Rucker.  These  sites  represent  a  variety  of  backgrounds: 

•  Drop  Zone — a  Yuma  sandy/silt  area  that  is  clear  of  vegetation  (this  area  is  the 
practice  drop  zone  for  parachute  training) 

•  Indian  Wash — a  Yuma  sandy/silt  area  with  small  foliage  (contains  desert 
varnish) 

•  Mitri  Lake— a  Yuma  area  that  is  a  bank  on  Mitri  Lake  with  dense  vegetation 

•  Pavement  Dyno — a  Yuma  area  that  is  a  pavement-covered  area 

•  Sand  Dyno — a  Yuma  area  that  corresponds  to  a  very  loose  sand  area’^ 

•  Plowed  Field— a  Fort  Rucker  silt  area  that  is  a  newly  plowed  field 

The  REMIDS  team  estimates  that  the  overflown  area  was  about  96  percent  of  the  total  area.  If  we 
assume  a  random  distribution  of  targets,  then  we  might  reasonably  expect  a  Pj  of  0.76/0.96  = 
79  percent,  relative  to  the  overflown  area. 

15  This  sand  is  not  indigenous  to  the  area  and  is  not  the  same  sand  as  that  appearing  in  Table  II-2. 


Table  III-2.  Background  False  Alarms  per  km*  Using  Stages  1  and  2  Only 
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0 
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Note:  PI  =  plastic 

WP  =  white  paint 

Al  =  aluminum 

FeOD  =  oxidized  iron  and  olive  drab  paint 


•  New  Crop — a  Fort  Rucker  area  that  is  covered  with  new  crop  vegetation 

•  Short  Vegetation— a  Fort  Rucker  area  that  has  field  vegetation  shorter  than 
2  inches. 

For  each  site,  information  was  gathered  for  a  single  pass  only;  hence,  the  total  area 
covered  in  each  case  was  much  less  than  a  square  kilometer  (see  the  table).  Table  III-2 
was  generated  for  values  of  1.6, 1.0,  and  0.7.  The  target  types  are  plastic  (PI),  white 
paint  (WP),  aluminum  (Al),  and  oxidized  iron  and  olive  drab  paint  (FeOD). 

Table  III-2  indicates  that  the  REMIDS  algorithm  would  correctly  predict  half  of 
these  sites  to  be  completely  ordnance  free.  These  results,  which  are  consistent  with  the 
discussion  in  Section  II  and  the  performance  at  Fort  Rucker,  suggest  that  REMIDS  may 


be  very  successful  in  assessing  large  tracts  of  sandy  or  grassy  land,  with  only  minimal 
operator  participation. 

For  large  tracts  of  land  similar  to  the  four  sites  in  Table  III-2  showing  a  signifi¬ 
cant  number  of  false  alarms,  further  analysis  (Stage  3,  Operator  Discrimination)  may  be 
required.  On  the  other  hand,  in  the  case  of  the  Drop  Zone,  the  false  alarms  were  located 
almost  exclusively  around  the  perimeter  of  the  site.  Thus,  REMIDS  would  have  correctly 
declared  most  of  the  site  to  be  ordnance-free.'^  Furthermore,  the  false  alarms  in  the  white 
paint  class  in  the  Pavement  Dyno  were  caused  by  the  white  striping  on  the  pavement  and 
therefore  could  easily  be  eliminated  by  an  operator.  On  the  other  hand,  those  alarms  m 
the  oxidized  iron/olive  drab  class  in  the  Pavement  Dyno  would  not  be  so  easily  discarded: 
it  is  thought  that  they  were  generated  by  machinery  and  vehicle  parts,  because  a  garage 
station  was  located  near  the  site.  As  expected,  the  Indian  Wash  generated  false  alarms  m 
the  iron-oxide/olive  drab  class. '8  At  Mitri  Lake,  which  had  the  most  false  alarms,  the 
false  alarms  were  scattered  over  much  of  the  site;  it  is  believed  that  they  were  caused  by 
tree  bark  (the  paper-like  bark  of  birch  trees  probably  yields  a  polarization  return  similar 
to  plastic).  Clearance  of  large  tracts  of  land  similar  to  Mitri  Lake  would  probably  not  be 
possible  without  the  Stage  3  analysis. 

C.  LESSONS  LEARNED 

There  were  two  major  lessons  learned  during  the  testing  of  the  REMIDS  system. 
First,  complete  coverage  is  not  likely  to  be  achieved  without  a  global  positioning  system 
(GPS)-aided  flight  guidance  system.  At  Yuma,  the  helicopter  flew  over  only  about 
96  percent  of  the  site,  and  therefore,  a  of  100  percent  was  not  achievable.  There  are 
four  main  factors  that  determine  the  amount  of  site  coverage:  (1)  the  amount  of  overlap 
between  adjacent  passes,  (2)  the  direction  of  the  wind  gusts,  (3)  the  intensity  of  the  wind 
gusts,  and  (4)  pilot  reaction  time  to  the  wind  gusts.  An  increase  in  overlap  does  not 
guarantee  full  coverage  because  wind  gusts  can  divert  the  helicopter  off  course. 


16  The  only  way  in  which  operator  skill  would  be  introduced  in  this  application  would  be  in  determining 
the  3A^  +  1  parameters  for  the  spectral  filter. 

It  is  thought  that  the  false  alarms  located  around  the  perimeter  may  have  been  caused  by  man-made 
items  such  as  parachutes  or  food  container  refuse. 

18  Note  that  the  number  of  false  alarms  in  the  Indian  Wash  area  was  not  that  high.  This  is  because  the 
desert  varnish  mainly  collects  on  the  edges  of  a  wash  area  and  not  in  the  wash  itself;  the  helicopter 
passed  over  the  center  of  the  wash  area. 


Second,  it  is  necessary  to  correct  for  image  shifting  due  to  the  rotation  frequency 
of  the  helicopter  blades.  This  image  shifting  occurred  at  approximately  17  Hz;  however, 
the  frequency  was  not  consistent  enough  to  attempt  a  correction  based  directly  on  the 
frequency.  In  addition,  the  magnitude  of  the  shifts  varied  greatly  throughout  the  image 
from  0  to  5  or  6  pixels.  Therefore,  corrective  shifts  had  to  be  calculated  on  a  localized 
basis  from  the  images  itself.  Previous  experience  had  shown  that  line-to-line  correlation 
could  be  effective  in  correcting  these  types  of  problems.  By  shifting  one  of  two  adjacent 
lines  by  various  amounts  and  computing  the  correlation  of  each  shifted  line  with  the 
unshifted  line,  the  optimal  shift  between  the  two  lines  could  be  identified.  The  major 
drawback  of  this  approach  is  that  strong  linear  features  in  the  terrain  bias  the  correlations. 
For  example,  a  road  that  runs  diagonally  across  the  image  will  result  in  “corrections”  that 
cause  the  road  to  run  vertically  down  the  image,  drastically  distorting  the  image.  This 
problem  was  alleviated  by  normalizing  the  shifts  so  that  they  summed  to  zero  over  each 
20-line  segment  of  the  image.  The  steps  in  the  correction  algorithm  are  the  following: 

j  _  Calculate  the  correlation  between  each  line  and  the  preceding  line  with  shifts 
Of -2,  -1, 0,  1,  and  2  pixels. 

2.  Use  the  correlation  values  from  step  1  to  calculate  a  subpixel  shift  between 
each  pair  of  lines. 

3.  Calculate  the  average  line  shift  over  each  20-line  segment  of  imagery. 
Twenty-line  averages  were  used  because  the  number  of  lines  in  1  cycle  of  the 
vibration  was  approximately  20. 

4.  Normalize  the  shifts  for  each  line  by  subtracting  the  20-line  average  shift  for 
the  segment  containing  the  line.  This  yields  a  net  total  shift  of  0  for  each 
20-line  segment  and  removes  any  bias  of  the  terrain  in  one  direction. 

5.  Apply  the  normalized  shifts  to  the  image. 

Figure  II-3(a)  shows  a  sample  image  before  the  correction,  and  Figure  II-3(b) 
shows  the  same  image  after  correction. 

D.  LIMITATIONS  OF  THIS  SYSTEM 

A  key  limitation  of  the  REMIDS  system  is  that  its  performance  is  highly  site 
dependent.  The  grassy  background  at  Fort  Rucker  enabled  the  system  to  easily  pick  out 
the  targets  based  on  their  spectral  information  alone;  no  shape  filter  was  needed.  On  the 
other  hand,  the  desert  varnish  at  Yuma  rendered  the  spectral  information  insufficient. 
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Figure  ll-3(a).  A  Sample  Image  before  the  Correction 


Figure  ll-3{b).  A  Sample  Image  after  the  Correction 
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It  became  apparent  from  the  Yuma  data  that  the  shape  filter  and  especially  the  target- 
recognition  capabilities  of  the  operator  will  play  a  critical  role  in  this  technology  for  any 
backgrounds  with  spectral  returns  close  to  those  of  the  targets  of  interest. 

In  addition  to  the  fact  that  some  backgrounds  render  discrimination  based  solely 
upon  spectral  information  impossible,  there  are  other  physical  limitations  of  the  system. 
For  this  system  to  work,  the  laser  must  have  a  line  of  sight  to  the  surface  ordnance  items. 
Thus,  broad-leaf  vegetation,  trees,  and  understory  can  pose  a  laser-penetration  problem. 
Dust-covered  ordnance  may  also  be  difficult  to  see;  even  if  the  dust  covering  is  such  that 
some  of  the  surface  is  exposed,  the  classification  of  the  UXO  may  be  difficult.  Finally, 
the  system  cannot  be  used  when  the  targets  are  covered  with  snow. 

The  REMIDS  system  is  more  effective  if  samples  of  materials  expected  at  the  site 
are  available  to  calibrate  the  spectral  classes.  The  polarization  returns  for  painted  objects 
can  vary  greatly  depending  upon  the  type  of  paint,  for  example.  Also,  the  polarization 
and  reflectance  returns  will  vary  with  ordnance  age.  For  some  material  types,  this  is  not  a 
big  problem;  rust,  for  example,  still  yields  a  fairly  high  polarization  return.  However,  the 
polarization  and  reflectance  returns  of  oxidized  brass  are  much  lower  than  those  of  new 
brass,  and  this  caused  some  problems  at  Yuma.  Specifically,  snake-eye  fin  assemblies 
that  were  at  the  Yuma  site  were  made  of  painted  brass.  The  nonweathered  fin  was  an  easy 
item  to  detect,  but  the  weathered  fin  was  not  associated  with  any  of  the  material  classes 
and  hence  went  undetected. 

If  this  system  were  used  at  a  site  where  only  limited  information  about  the 
expected  ordnance  items  was  available,  then  the  performance  would  likely  be  below  that 
exhibited  at  Fort  Rucker  and  at  Yuma.  In  principle,  one  could  try  to  include  as  many 
material  classes  as  possible;  there  is  no  limit  to  the  number  of  material  classes  used  in 
the  spectral  filter.  In  practice,  however,  the  more  classifications,  the  more  complex  the 
processing,  since  the  total  number  of  free  parameters  used  in  the  spectral  filter  is  3A  +  1, 
where  N  is  the  number  of  spectral  classifications.  Hence,  to  optimize  the  performance  of 
this  system,  one  should  find  out  as  much  as  possible  about  the  UXO  at  the  site  in 
question,  particularly  the  material  composition  and  age  of  the  ordnance. 

E.  CONCLUSIONS 

Based  on  these  results,  it  is  clear  that  the  performance  of  this  system  is  site 
dependent.  If  the  background  is  grassy  as  at  Fort  Rucker,  it  is  quite  likely  that  a  Pj  of  at 
least  90  percent  is  achievable  with  very  low  false-alarm  rates  using  only  the  Stage  1 


analysis.  At  sites  such  as  Yuma,  such  performance  is  probably  impossible  with  just  the 
spectral  information.  However,  fair-to-good  performance  can  be  expected  once  the  size/ 
shape  filter  and  operator  filter  are  employed,  if  one  is  willing  to  accept  that  the  system 
cannot  be  used  to  detect  all  ordnance.  Specifically,  if  there  is  a  high  probability  of  large 
numbers  of  false  alarms,  a  minimum  size  requirement  will  probably  be  necessary,  and  the 
system  will  therefore  not  be  able  to  detect  items  smaller  than  this  minimum  size.  Thus,  if 
REMIDS  were  used  to  detect  surface  UXO  larger  than  4  pixels  at  a  site  such  as  Yuma,  a 
Pj  of  about  76  percent  with  a  very  low  FAR  might  be  expected,  assuming  a  declaration 
radius  of  5  m.  (We  note  again  that  this  number  is  probably  a  lower-bound  estimate  of  Pj, 
because  holiday  areas  were  not  accounted  for  in  the  calculation.) 

On  the  other  hand,  in  realistic  scenarios  with  large  areas,  the  methodology 
adopted  at  Yuma  would  have  to  be  modified,  because  too  much  emphasis  was  placed  on 
the  ability  of  the  operator  to  discriminate  the  ordnance  items.  This  modification  would 
entail  a  more  sophisticated  shape/size  filter  to  pre-screen  the  objects,  so  that  the  operator 
could  complete  his  role  in  a  reasonable  amount  of  time.  It  is  not  known  whether 
transferring  the  bulk  of  the  discrimination  from  the  operator  to  the  size/shape  filter  will 
result  in  similar  performance  as  that  achieved  at  Yuma,  although  it  is  likely  that  at  least 
some  degradation  in  performance  may  be  expected  (see  discussion  in  II.B.3). 

Although  we  cannot  state  for  certain  whether  this  system  can  be  relied  upon  to 
predict  individual  ordnance  types  with  high  Pj  and  low  Pj^  over  large  areas  with 
potentially  challenging  backgrounds,  we  do  feel  reasonably  confident  in  claiming  that 
this  system  has  potential  in  a  large-area  search  mode  of  operation,  with  minimal  operator 
participation,  especially  for  grassy  or  sandy  sites.  It  is  often  the  case  that  when  large 
tracts  of  land  must  be  cleared  of  UXO,  a  significant  portion  of  that  land  is  already  clean, 
but  it  is  not  known  which  portions  are  clean  and  which  must  be  cleared.  From  the 
discussion  in  Section  III.B,  it  is  evident  that  the  REMIDS  system  could  play  a  valuable 
role  in  clearing  large  tracts  of  land  by  determining  quickly  which  areas  are  already  likely 
to  be  clean  and  which  warrant  further  investigation.  This  analysis  can  be  performed 
solely  with  the  spectral  filter  and  the  crude  size  filter  employed  at  Yuma.  More  sophis¬ 
ticated  size/shape  filters  will  render  this  system  even  more  effective  in  screening  large 
areas.  In  any  case,  because  the  spectral  and  shape/size  filters  do  not  require  much 
operator  input,  this  screening  can  be  performed  quickly  and  reliably . 
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IV.  COST  ASSESSMENT 


The  following  estimate  includes  the  cost  of  transporting  the  REMIDS  system  to 
and  from  the  site  in  addition  to  the  costs  of  collecting  18  hours  of  flight  data. 


1.  Equipment  installation  and  calibration . $95,000 

2.  Helicopter  Support  (includes  ferry  hours  to  transport 

the  helicopter  and  its  crew) . $120,000 

3 .  Ground  Equipment  Transportation,  Setup,  and  Rental . $  1 8,000 

4.  Data  collection  (includes  the  cost  of  flying  the  helicopter 

and  the  crew  for  18  hours) . $48,000 

5.  Data  analysis  on  site  (roughly  3  man-months,  includes 

per  diem  and  transportation) . $68,000 

Total  $349,000 


The  first  item  includes  the  installation  of  the  flight-aided  digital  global  positioning 
system  (DGPS).  The  use  of  regular  DGPS  will  reduce  the  cost  of  the  first  item  to  $50,000 
but  it  will  increase  the  fourth  item  to  $70,000,  for  an  overall  cost  of  $326,000.  The  lack 
of  flight-aided  DGPS  will  limit  the  ability  to  cover  areas  missed  on  the  initial  flyover 
(holiday  areas),  and  it  will  also  necessitate  the  use  of  visual  markers  on  the  ground  for  the 
desired  flight  lines. 

The  area  that  can  be  covered  in  18  flight  hours  will  depend  upon  the  profile  of  the 
data  collection  flights  (i.e.,  altitude  and  forward  speed),  as  well  as  the  chosen  flight  path. 
For  example,  if  the  0.5  km  x  1  km  site  at  Yuma  is  traversed  crosswise  (i.e.,  using  500-m 
passes),  then  during  1  flight  hour,  only  about  15  minutes  is  dedicated  to  actual  data  col¬ 
lection.  The  rest  of  the  time  is  spent  making  turns  and  lining  up  for  the  next  pass.  In  this 
case,  with  a  pixel  resolution  of  1.9  in.  x  1.9  in.,  1  flight  hour  yields  about  100  acres  of 
coverage.  On  the  other  hand,  if  the  site  is  traversed  lengthwise,  the  coverage  will  be  much 
greater;  estimates  provided  by  the  REMIDS  team  are  on  the  order  of  300  acres  per  flight 
hour.  A  lower  resolution  will  also  allow  for  more  coverage  per  flight  hour;  for  example, 
decreasing  the  resolution  to  4.5  in.  x  4.5  in.  will  yield  about  a  factor  of  5  increase  in  area 
coverage  per  flight  hour. 


In  addition  to  the  above,  a  cost  estimate  for  this  system  should  include  the  time 
for  operator  training.  The  developers  of  this  system  estimate  that  at  least  3  man-months 
will  be  needed  per  operator. 
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